Skip to content

Generalize LLM post-processing to any OpenAI-API-compatible server#57

Draft
JRufer wants to merge 7 commits into
masterfrom
claude/adoring-curie-s2uzrk
Draft

Generalize LLM post-processing to any OpenAI-API-compatible server#57
JRufer wants to merge 7 commits into
masterfrom
claude/adoring-curie-s2uzrk

Conversation

@JRufer

@JRufer JRufer commented Jun 23, 2026

Copy link
Copy Markdown
Owner

Renames the Ollama settings tab to "OpenAI API" and generalizes the LLM client to use OpenAI-compatible endpoints (/v1/models, /v1/chat/completions) plus an optional API key field. See commit message for details.

🤖 Generated with Claude Code


Generated by Claude Code

claude added 7 commits June 23, 2026 18:12
Rename the settings tab from "Ollama" to "OpenAI API" and switch the
LLM client from Ollama's native API to the OpenAI-compatible
/v1/models and /v1/chat/completions endpoints, so any compatible
server (Ollama, LM Studio, OpenAI, etc.) can be used. The server URL
still defaults to localhost but remains user-editable, and a new
optional API key field is sent as an Authorization bearer token for
non-local servers.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PThnjsJtj5TRZNBzCLECoi
…_DISPLAY

test_clipboard_target_linux_cli temporarily mutates the process-wide PATH
and WAYLAND_DISPLAY env vars to mock wl-copy. Running concurrently with
test_clipboard_target_success let the latter observe the mock during
deliver() but not during the follow-up test() call once the other test
had restored PATH, causing a flaky "assertion failed: test_res.reachable"
failure in CI. Serialize both tests with a mutex.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PThnjsJtj5TRZNBzCLECoi
Lets a hotkey deliver transcribed text to any OpenAI-compatible chat
completions endpoint and route the model's response (e.g. a summary or
rewrite) to the target's destination, instead of just the raw transcript.
Reuses the global OpenAI API connection settings (endpoint/api_key/model)
with optional per-target overrides for model, max_tokens, and timeout.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PThnjsJtj5TRZNBzCLECoi
Config is wrapped (state.config.lock().await.data.*), not flat;
the callback registered for the new OpenAI API target referenced
.ollama directly instead of .data.ollama, breaking the build.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PThnjsJtj5TRZNBzCLECoi
cargo test failed to compile voxctrl-app because two OutputTarget
struct literals in the sequential-delivery test predated the new
openai_* fields.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PThnjsJtj5TRZNBzCLECoi
Each openai_api target now keeps an in-memory chat log (last 20
messages / 10 exchanges) and replays it with every request, so
follow-up dictations like "what did I just ask you?" resolve
correctly instead of being treated as an isolated one-shot prompt.
History lives only in the running process and resets on hot-reload
or restart.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PThnjsJtj5TRZNBzCLECoi
The earlier openai_api target history fix didn't help users whose Q&A
flow goes through the legacy ollama_enabled pre-processing pipeline
(OllamaClient::process), which was completely stateless. Add a
process_with_history method and a per-binding in-memory history map on
InferenceEngine so follow-up questions like "what did I just ask you"
resolve correctly for that pipeline too.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants